Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee $\&$ Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)\footnote{source code: https://github.com/wh1992v/ggp-rl}, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the $\epsilon$-greedy strategy, we propose a first enhancement, the dynamic $\epsilon$ algorithm. In addition, inspired by (Gelly $\&$ Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Comment: arXiv admin note: substantial text overlap with arXiv:1802.0594

Drag on a partially immersed sphere at the capillary scale

We study the drag on a centimetric sphere in a uniform flow in the presence of a free surface as a function of submergence depth. Through direct force measurements in a custom benchtop recirculating flume, we demonstrate that the drag can significantly exceed the corresponding drag in a single-phase flow and achieves a peak at submergence depths just prior to complete immersion. The additional drag in the partially immersed state is rationalized by considering hydrostatic effects associated with the asymmetric surface height profile induced by the obstacle in the flow direction which persists for flow speeds below the minimum capillary-gravity wave speed. At these scales, the sphere's wettability plays a pronounced role in determining the maximum possible drag and results in hysteretic behaviors near touchdown and complete immersion. The influence of flow speed, sphere size, and surface tension on the drag characteristics are additionally explored through a combination of experiments and numerical simulations.Comment: 9 figure

The Detection of Ionizing Radiation by Plasma Panel Sensors: Cosmic Muons, Ion Beams and Cancer Therapy

The plasma panel sensor is an ionizing photon and particle radiation detector derived from PDP technology with high gain and nanosecond response. Experimental results in detecting cosmic ray muons and beta particles from radioactive sources are described along with applications including high energy and nuclear physics, homeland security and cancer therapeuticsComment: Presented at SID Symposium, June 201

A new set of BXD recombinant inbred lines from advanced intercross populations in mice

BACKGROUND: Recombinant inbred (RI) strains are an important resource for mapping complex traits in many species. While large RI panels are available for Arabidopsis, maize, C. elegans, and Drosophila, mouse RI panels typically consist of fewer than 30 lines. This is a severe constraint on the power and precision of mapping efforts and greatly hampers analysis of epistatic interactions. RESULTS: In order to address these limitations and to provide the community with a more effective collaborative RI mapping panel we generated new BXD RI strains from two independent advanced intercrosses (AI) between C57BL/6J (B6) and DBA/2J (D2) progenitor strains. Progeny were intercrossed for 9 to 14 generations before initiating inbreeding, which is still ongoing for some strains. Since this AI base population is highly recombinant, the 46 advanced recombinant inbred (ARI) strains incorporate approximately twice as many recombinations as standard RI strains, a fraction of which are inevitably shared by descent. When combined with the existing BXD RI strains, the merged BXD strain set triples the number of previously available unique recombinations and quadruples the total number of recombinations in the BXD background. CONCLUSION: The combined BXD strain set is the largest mouse RI mapping panel. It is a powerful tool for collaborative analysis of quantitative traits and gene function that will be especially useful to study variation in transcriptome and proteome data sets under multiple environments. Additional strains also extend the value of the extensive phenotypic characterization of the previously available strains. A final advantage of expanding the BXD strain set is that both progenitors have been sequenced, and approximately 1.8 million SNPs have been characterized. This provides unprecedented power in screening candidate genes and can reduce the effective length of QTL intervals. It also makes it possible to reverse standard mapping strategies and to explore downstream effects of known sequence variants

Perinatal assessment of complex cesarean delivery: beyond placenta accreta spectrum

Multiple cesarean deliveries are known to be associated with long-term postoperative consequences because of a permanent defect of the lower uterine segment wall and the development of thick pelvic adhesions. Patients with a history of multiple cesarean deliveries often present with large cesarean scar defects and are at heightened risk in subsequent pregnancies of cesarean scar ectopic pregnancy, uterine rupture, low-lying placenta or placenta previa, and placenta previa accreta. Moreover, large cesarean scar defects will lead to progressive dehiscence of the lower uterine segment with the inability to effectively reapproximate hysterotomy edge and repair at birth. Major remodeling of the lower uterine segment associated with true placenta accreta spectrum at birth, whereby the placenta becomes inseparable from the uterine wall, increases the rates of perinatal morbidity and mortality, especially when undiagnosed before delivery. Ultrasound imaging is currently not routinely used to evaluate the surgical risks of patients with a history of multiple cesarean deliveries, beyond the risk assessment of placenta accreta spectrum. Independent of accreta placentation, a placenta previa under a scarred, thinned partially disrupted lower uterine segment, covered by thick adhesions with the posterior wall of the bladder, poses a surgical risk and requires fine dissection and surgical expertise; however, data on the use of ultrasound to evaluate uterine remodeling and adhesions between the uterus and other pelvic organs are scarce. In particular, transvaginal sonography has been underused, including in patients with a high probability of placenta accreta spectrum at birth. Based on the best available knowledge, we discuss the role of ultrasound imaging in identifying the signs suggestive of major remodeling of the lower uterine segment and in mapping the changes in the uterine wall and pelvis, to enable the surgical team to prepare for all different types of complex cesarean deliveries. The need for postnatal confirmation of the prenatal ultrasound findings for all patients with a history of multiple cesarean deliveries, regardless of the diagnosis of placenta previa and placenta accreta spectrum, is discussed. We propose an ultrasound imaging protocol and a classification of the level of surgical difficulty at elective cesarean delivery to stimulate further research toward the validation of ultrasound signs by which these signs may be applied to improve surgical outcomes

Fibroblast and Epidermal Cell-Type I Collagen Interactions: Cell Culture and Human Studies

Fibroblast and epidermal cell-type I collagen sponge interactions were studied in cell culture as well as in humans. In cell culture, fibroblasts were observed to migrate and proliferate throughout a type I collagen sponge containing either hyaluronic acid (HA) or fibronectin (FN). Fibroblasts accumulated in the center of the pores in sponges containing HA and appeared to surround themselves with newly synthesized extracellular matrix. In sponges containing FN, fibroblasts attached to and elongated along the collagen fibers of the sponge. In the absence of FN or HA protein synthesis of fibroblasts appeared to be inhibited by the presence of the type I collagen sponge. Epidermal cells grown on plastic or on type I collagen, formed sheets. Epidermal cells grown on a collagen sponge morphologically appeared different than cells grown on plastic. The type I collagen matrix studied in cell culture was applied to dermal wounds of patients with pressure ulcers in order to evaluate its effect on dermal wound healing. The areas of ulcers treated for 6 weeks with a type I collagen sponge decreased by about 40% compared with no change in the areas of untreated controls. Preliminary results suggest that a type I collagen sponge is a biocornpatible substrate with fibroblasts and epidermal cells and may be effective in enhancing healing of chronic skin ulcers

FIGO consensus guidelines on placenta accreta spectrum disorders : Epidemiology

Peer reviewe

Sleep During Pregnancy: The nuMoM2b Pregnancy and Sleep Duration and Continuity Study

Study Objectives: To characterize sleep duration, timing and continuity measures in pregnancy and their association with key demographic variables. Methods: Multisite prospective cohort study. Women enrolled in the nuMoM2b study (nulliparous women with a singleton gestation) were recruited at the second study visit (16-21 weeks of gestation) to participate in the Sleep Duration and Continuity substudy. Women <18 years of age or with pregestational diabetes or chronic hypertension were excluded from participation. Women wore a wrist activity monitor and completed a sleep log for 7 consecutive days. Time in bed, sleep duration, fragmentation index, sleep efficiency, wake after sleep onset, and sleep midpoint were averaged across valid primary sleep periods for each participant. Results: Valid data were available from 782 women with mean age of 27.3 (5.5) years. Median sleep duration was 7.4 hours. Approximately 27.9% of women had a sleep duration of 9 hours. In multivariable models including age, race/ethnicity, body mass index, insurance status, and recent smoking history, sleep duration was significantly associated with race/ethnicity and insurance status, while time in bed was only associated with insurance status. Sleep continuity measures and sleep midpoint were significantly associated with all covariates in the model, with the exception of age for fragmentation index and smoking for wake after sleep onset. Conclusions: Our results demonstrate the relationship between sleep and important demographic characteristics during pregnancy

Prescription and Other Medication Use in Pregnancy

OBJECTIVE: To characterize prescription and other medication use in a geographically and ethnically diverse cohort of women in their first pregnancy. METHODS: In a prospective, longitudinal cohort study of nulliparous women followed through pregnancy from the first trimester, medication use was chronicled longitudinally throughout pregnancy. Structured questions and aids were used to capture all medications taken as well as reasons they were taken. Total counts of all medications taken including number in each category and class were captured. Additionally, reasons the medications were taken were recorded. Trends in medications taken across pregnancy and in the first trimester were determined. RESULTS: Of the 9,546 study participants, 9,272 (97.1%) women took at least one medication during pregnancy with 9,139 (95.7%) taking a medication in the first trimester. Polypharmacy, defined as taking at least five medications, occurred in 2,915 (30.5%) women. Excluding vitamins, supplements, and vaccines, 73.4% of women took a medication during pregnancy with 55.1% taking one in the first trimester. The categories of drugs taken in pregnancy and in the first trimester include the following: gastrointestinal or antiemetic agents (34.3%, 19.5%), antibiotics (25.5%, 12.6%), and analgesics (23.7%, 15.6%, which includes 3.6%; 1.4% taking an opioid pain medication). CONCLUSION: In this geographically and ethnically diverse cohort of nulliparous pregnant women, medication use was nearly universal and polypharmacy was common

The future of human nature: a symposium on the promises and challenges of the revolutions in genomics and computer science, April 10, 11, and 12, 2003

This repository item contains a single issue of the Pardee Conference Series, a publication series that began publishing in 2006 by the Boston University Frederick S. Pardee Center for the Study of the Longer-Range Future. This was the Center's Symposium on the Promises and Challenges of the Revolutions in Genomics and Computer Science took place during April 10, 11, and 12, 2003. Co-organized by Charles DeLisi and Kenneth Lewes; sponsored by Boston University, the Frederick S. Pardee Center for the Study of the Longer-Range Future.This conference focused on scientific and technological advances in genetics, computer science, and their convergence during the next 35 to 250 years. In particular, it focused on directed evolution, the futures it allows, the shape of society in those futures, and the robustness of human nature against technological change at the level of individuals, groups, and societies. It is taken as a premise that biotechnology and computer science will mature and will reinforce one another. During the period of interest, human cloning, germ-line genetic engineering, and an array of reproductive technologies will become feasible and safe. Early in this period, we can reasonably expect the processing power of a laptop computer to exceed the collective processing power of every human brain on the planet; later in the period human/machine interfaces will begin to emerge. Whether such technologies will take hold is not known. But if they do, human evolution is likely to proceed at a greatly accelerated rate; human nature as we know it may change markedly, if it does not disappear altogether, and new intelligent species may well be created